In this work, we introduce a control framework that combines model-based footstep planning with Reinforcement Learning (RL), leveraging desired footstep patterns derived from the Linear Inverted Pendulum (LIP) dynamics. Utilizing the LIP model, our method forward predicts robot states and determines the desired foot placement given the velocity commands. We then train an RL policy to track the foot placements without following the full reference motions derived from the LIP model. This partial guidance from the physics model allows the RL policy to integrate the predictive capabilities of the physics-informed dynamics and the adaptability characteristics of the RL controller without overfitting the policy to the template model. Our approach is validated on the MIT Humanoid, demonstrating that our policy can achieve stable yet dynamic locomotion for walking and turning. We further validate the adaptability and generalizability of our policy by extending the locomotion task to unseen, uneven terrain. During the hardware deployment, we have achieved forward walking speeds of up to 1.5 m/s on a treadmill and have successfully performed dynamic locomotion maneuvers such as 90-degree and 180-degree turns.

This paper proposes a novel orientation-aware model predictive control (MPC) for dynamic humanoid walking that can plan footstep locations online. Instead of a point-mass model, this work uses the augmented single rigid body model (aSRBM) to enable the MPC to leverage orientation dynamics and stepping strategy within a unified optimization framework. With the footstep location as part of the decision variables in the aSRBM, the MPC can reason about stepping within the kinematic constraints. A task-space controller (TSC) tracks the body pose and swing leg references output from the MPC, while exploiting the full-order dynamics of the humanoid. The proposed control framework is suitable for real-time applications since both MPC and TSC are formulated as quadratic programs. Simulation investigations show that the orientation-aware MPC-based framework is more robust against external torque disturbance compared to state-of-the-art controllers using the point mass model, especially when the torso undergoes large angular excursion. The same control framework can also enable the MIT Humanoid to overcome uneven terrains, such as traversing a wave field.

-- Human beings can utilize multiple balance strategies, e.g. In this work, we propose a novel Nonlinear Model Predictive Control (NMPC) framework for robust locomotion, with the capabilities of step location adjustment, Center of Mass (CoM) height variation, and angular momentum adaptation. These features are realized by constraining the Zero Moment Point within the support polygon. By using the nonlinear inverted pendulum plus flywheel model, the effects of upper-body rotation and vertical height motion are considered. As a result, the NMPC is formulated as a quadratically constrained quadratic program problem, which is solved fast by sequential quadratic programming. Using this unified framework, robust walking patterns that exploit reactive stepping, body inclination, and CoM height variation are generated based on the state estimation. The adaptability for bipedal walking in multiple scenarios has been demonstrated through simulation studies. Humanoid robots have attracted much attention for their capabilities in accomplishing challenging tasks in real-world environments. With several decades passed, state-of-the-art robot platforms such as ASIMO [1], Atlas [2], W ALK-MAN [3], and CogIMon [4] have been developed for this purpose. However, due to the complex nonlinear dynamics of bipedal locomotion over the walking process, enhancing walking stability, which is among the prerequisites in making humanoids practical, still needs further studies. In this paper, inspired by the fact that human beings can make use of the redundant Degree of Freedom (DoF) and adopt various strategies, such as the ankle, hip, and stepping strategies, to realize balance recovery [5]-[7], we aim to develop a versatile and robust walking pattern generator which can integrate multiple balance strategies in a unified way. To generate the walking pattern in a time-efficient manner, simplified dynamic models have been proposed, among which the Linear Inverted Pendulum Model (LIPM) is widely used [8]. Using the LIPM, Kajita et al. proposed the preview control for Zero Moment Point (ZMP) tracking [9]. By adopting a Linear Quadratic Regulator (LQR) scheme, the ankle torque was adjusted to modulate the ZMP trajectory and Center of Mass (CoM) trajectory. Nevertheless, this strategy can neither modulate the step parameters nor take into consideration the feasibility constraints arisen from actuation limitations and environmental constraints. To overcome this drawback, Wieber et al. proposed a Model Predictive Control (MPC) algorithm to utilize the ankle strategy [10] and then extended it for adjusting step location [11].